A Proof of Orthogonal Double Machine Learning with Z-Estimators

We consider two stage estimation with a non-parametric first stage and a generalized method of moments second stage, in a simpler setting than [CCD16]. We give an alternative proof of the theorem given in Chernozhukov et al. [CCD16] that orthogonal second stage moments, sample splitting and n-consistency of the first stage, imply √ n-consistency and asymptotic normality of second stage estimates. Our proof is for a variant of their estimator, which is based on the empirical version of the moment condition (Z-estimator), rather than a minimization of a norm of the empirical vector of moments (M-estimator). This note is meant primarily for expository purposes, rather than as a new technical contribution. 1 Two-Stage Estimation Suppose we have a model which predicts the following set of moment conditions: E[m(Z, θ0, h0(X))] = 0 (1) where θ0 ∈ R is a finite dimensional parameter of interest, h0 : S → R is a nuisance function we do not know, Z are the observed data which are drawn from some distribution and X ∈ S is a subvector of the observed data. We want to understand the asymptotic properties of the following two-stage estimation process: 1. First stage. Estimate h0(·) from an auxiliary data set (e.g. running some non-parametric regresssion) yielding an estimate ĥ. 2. Second stage. Use the first stage estimate ĥ and compute an estimate θ̂ of θ0 from an empirical version of the moment condition: i.e.